So, you've broken your input into a stream of tokens.

这样，您已经将输入拆分为一连串的记号。

youdao

The power of thought was restored; he knew that the rope had broken and he had fallen into the stream.

一切归于寒冷和黑暗。思考的能力逐渐恢复过来；他知道，绳子断了，自己落入河水中。

youdao

Album sales have plummeted by more than half since 2000, taking away a lucrative earnings stream for musicians and one of the ways that artists have traditionally broken through into the mainstream.

2000年以来，音乐专辑销量暴跌了一半以上，歌手们少了一个赚大钱的路子，也使艺人们少了一个进入主流的传统方式。

youdao

Urgent cases, like broken limbs, come in a steady stream.

FORBES: Magazine Article